Escaping Strict Saddle Points of the Moreau Envelope in Nonsmooth Optimization
نویسندگان
چکیده
Recent work has shown that stochastically perturbed gradient methods can efficiently escape strict saddle points of smooth functions. We extend this body to nonsmooth optimization, by analyzing an inexact analogue a method applied the Moreau envelope. The main conclusion is variety algorithms for optimization envelope at controlled rate. technical insight many proximal subproblem yield directions approximate
منابع مشابه
A Generic Approach for Escaping Saddle points
A central challenge to using first-order methods for optimizing nonconvex problems is the presence of saddle points. First-order methods often get stuck at saddle points, greatly deteriorating their performance. Typically, to escape from saddles one has to use second-order methods. However, most works on second-order methods rely extensively on expensive Hessian-based computations, making them ...
متن کاملEfficient approaches for escaping higher order saddle points in non-convex optimization
Local search heuristics for non-convex optimizations are popular in applied machine learning. However, in general it is hard to guarantee that such algorithms even converge to a local minimum, due to the existence of complicated saddle point structures in high dimensions. Many functions have degenerate saddle points such that the first and second order derivatives cannot distinguish them with l...
متن کاملEscaping From Saddle Points - Online Stochastic Gradient for Tensor Decomposition
We analyze stochastic gradient descent for optimizing non-convex functions. In many cases for non-convex functions the goal is to find a reasonable local minimum, and the main concern is that gradient updates are trapped in saddle points. In this paper we identify strict saddle property for non-convex problem that allows for efficient optimization. Using this property we show that from an arbit...
متن کاملNonsmooth interval-valued optimization and saddle-point optimality criteria
In this article, we focus our attention on a nonsmooth interval-valued optimization problem and establish sufficient optimality conditions for a feasible solution to be a LU optimal solution under the invexity assumption. Appropriate duality theorems for Wolfe and Mond-Weir type duals are presented in order to relate the LU optimal solution of primal and dual programs. Moreover, saddle-point ty...
متن کاملBowl breakout: escaping the positive region when searching for saddle points.
We present a scheme improving the minimum-mode following method for finding first order saddle points by confining the displacements of atoms to the subset of those subject to the largest force. By doing so it is ensured that the displacement remains of a local character within regions where all eigenvalues of the Hessian matrix are positive. However, as soon as a region is entered where an eig...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Siam Journal on Optimization
سال: 2022
ISSN: ['1095-7189', '1052-6234']
DOI: https://doi.org/10.1137/21m1430868